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Abstract 

Kolmogorov introduced a concept of e-entropy to analyze informa- 
tion in classical continuous system. The fractal dimension of geomet- 
rical sets was introduced by Mandelbrot as a new criterion to analyze 
the complexity of these sets. The e-entropy and the fractal dimension 
of a state in general quantum system were introduced by one of the 
present authors in order to characterize chaotic properties of general 
states. 

In this paper, we show that e-entropy of a state includes Kol- 
mogorov e-entropy, and the fractal dimension of a state describe frac- 
tal structure of Gaussian measures. 

1 Introduction 

The £-entropy was introduced by Kolmogorov (1963) using the mutual en- 
tropy with respect to two random variables / and g. The entropy <S'(/) of a 
random variable / is usually infinite on a continuous probability space. On 
the other hand, the e-entropy >S'Koimogorov(/; ( 'S'k(/;£) for short ) can be 
bounded. Therefore, we can use the £ -entropy to analyze random variables 
in classical system. This e-entropy S'k(/; e) expresses a degree of information 
transmission in the e-neighborhood of a random variable /. 

By the way, Mandelbrot introduced a new criterion to analyze complexity 
of geometrical sets, it is so called fractal dimension (Mandelbrot, 1982), which 
is different from the euclidean dimensions. Usual fractal theory mostly treats 
only geometrical sets. It is desirable to extend the fractal dimensions in order 
to characterize some other objects. One of the present authors introduced 
the notion of e- entropy >S'ohya(A*; £) ('S'o(a*; ^) for short) for a state in order to 
formulate the fractal dimension of a state in general quantum system (GQS 
for short) (Ohya, 1989), (Ohya, 1991), (Ohya and Petz, 1993). Actually, the 
capacity dimensions , which is one of the fractal dimensions for geometrical 
sets, was given by the e-entropy. Namely, it is defined by 

logA^x(£) ... 
dc{X) = hm—^-^, (1) 

where Nx{£) is the minimum numbers of a convex set with diameter e cov- 
ering a set X and log Nx{s) is called e-entropy of a geometrical set X (Kol- 
mogorov and Tihomirov, 1961). The capacity dimension characterize fractal 
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structure of the geometrical sets as a hmiting behavior of e-e ntropy when s 
approach to 0. 

Our fractal dimension of a state is formulated by extending the concept 
of the capacity dimension to GQS. That is, the fractal dimension of a state is 
expressed by the e-entropy of a state instead of the e-entropy of a geometrical 
set X and characterize fractal structure of a state. 

These e-entropy and fractal dimension provide new criteria describing the 
complexity of states, so that they can be used to distinguish two states even 
when they have the same value for the entropy. For instance, we could analyze 
the complexity for some systems by these criteria (Ohya, 1991),(Matsuoka 
and Ohya, 1 995), (Akashi, 1992). 

In this paper, we examine the similarity and difference of two £-entropies 
Sk and 5*0 for Gaussian measures (states) on a Hilbert space. It is shown 
that our e -entropy and fractal dimension are useful to classify Gaussian 
measures, in the case of that Kolmogorov's e-entropy can not be used. 



2 Kolmogorov's e-entropy for random vari- 
ables 

In this section, we remind the definition of Kolmogorov's e -entropy for ran- 
dom variables. Let {fl, $5, //) be a probability space and M{fl) be the set of 
all random variables, and /, g be two random variables on Q with valued on 
a metric space {X,d). Let /x/ be a probability measure associated with a 
random variable /. Then the mutual entropy /(/, g) of the random variable 
/ and g is defined by (Gelfand and Yaglom, 1959) 

^{f,9) = S{lifg,Hf® fig) 

= I fxXX^^^Og^^^dfif0fig (fifg^flf0fig) ^ 

\ OO (otherwise) ' 

where S{-,-) is relative entropy (Kulback-Leibler information), fig is the 
direct product probability measure of / and g, and jifg is the joint probability 
distribution of / and g, lijif^iJ. Radon-Nikodym derivative of jifg with 

respect io jXf ® jXg. Moreover the entropy S{f) of the random variable / is 
given by 

S{f) = I{fJ). (2) 
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S{f ) is often infinite in continuous case. Kolmogorov introduced tlie e- en- 
tropy for a random variable / as follows; 

5k(/; e) = inf {/(/, gy,ge M,{f; e)} , (3) 

where 

Mdif; e)^i^ge M(Q) ; ^ J^^^dix,yrdf,f,ix,y) < sj . (4) 

3 e-entropy and fractal dimensions of a state 

The e-entropy and the fractal dimension of a state were introduced in Ohya 
(1991) for GQS. In this section, we review these formulations in the frame- 
work of classical measure theory. 

In the information theory, an input state is described by a state (probabil- 
ity measure in continuous classical system, density operator in usual quantum 
system) and it is sent to an output system (receiver) through some channel 
denoted by A*. A channel is a transmitter (e.g. optical fiber), mathemati- 
cally it is a mapping from an input state space to an output state space. 

When an input state dynamically changes to an output state p,{= A*/x) 
under a channel A*, we ask how much information carried by fj, can be trans- 
mitted to the output state through the channel A*. It is the mutual entropy 
that represents this amount of information transmitted from fj. to p,. Hence, 
the mutual entropy depends on an input state and a channel. In this scheme, 
the mutual entropy is formulated as follows: 

Let (Qi,9i) be an input space, {fl2,^2) be an output space and P{flk) 
be the set of all probability measures on {Qk, ^k){k = 1,2). We can call the 
following linear mapping A* from P{Qi) to P{fl2) a. channel (Markov kernel): 

HQ) = A>(g) 

= / \{uj,Q)dn{u) //eP(Qi), 

J Hi 

where A is a mapping from Oi x to [0, 1] satisfying the following conditions: 



(1) A(-, Q) is a measurable function on VLi for each Q G 3^2- 
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(2) X{lo, ...) e P{n2) for each u; e fli. 



The compound state $ of and jj, is given by 

$(QixQ2)=/ X{cu,Q2) dnicu), (5) 

for any QiG^^i, Q2G^^2- The mutual entropy in classical continuous system 
can be expressed by the relative entropy S{-, •) of the compound state $ and 
the direct product state $0 = ® A*/Lt: 



/(/.;A*)=5($,$o) (= /(a^,/^;*)) 

(6) 



00 (otherwise) ' 

where ^ is the Radon-Nikodym derivative of $ with respect to $o- 

In the following discussion, we consider the case of = (Q2,Q'2) = 

(^,9") for simplicity. 

We shall give the definition of Kolmogorov's £-entropy S'k(/u;£) for a 

general probability measure \i on (f], 

5k(//; e) = inf {I{fx, jl;<^) ; \\fx - < e} , (7) 

where ||^|| is a certain norm of /i. 

The £-entropy of a state fi is defined as follows (Ohya, 1989). 

Definitionl : (The s-entropy of a state fi E P (Q) ) 

Soifi; e) = inf {J{fx; A*); - A>|| < e} , (8) 

A* 

where is a certain norm of and 

J{f,; A*) = sup {/(//; P); T// = A>} , (9) 

p* 

Here J(^; A*) is called the maximum mutual entropy w.r.t. fi and A*. 

The £-entropy S'o(yU;£) is a bit more general than the Kolmogorov e- 
entropy 5'k(/; s) for random variables, more precisely the e-entropy Soi/J'', s) 
is different from 5'k(a*;£) in the following points. 
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(1) The definition is based on states (probability measures) not only ran- 
dom variables. 

(2) Several possibilities to choose the norm of states. 

(3) The concept of the maximum mutual entropy J{n; A*) is used, which 
is a very essential as is described in section 6. 

This fractal dimension of a state in a classical continuous system is 
defined by the e-entropy of states. 

Definition2: (The capacity dimension of a state n e P {^) ) 

4 Gaussian measure and Gaussian channel on 
a Hilbert Space 

We briefly review the Gaussian communication processes treated by Baker 
et al (Baker, 1978),(Yanagi, 1988). 

Let B be the Borel a-field of a real separable Hilbert space Ti and /i be a 
Borel probability measure on B satisfying 



x|| dii{x) < oo. (11) 

H " 

Further, we denote the set of all positive self-adjoint trace class operators 
on n by T{H)+ {= {R G B{n)] R>0, R = R^tiR < oo}) and define the 
mean vector m^G 7i and the covariance operator Rf^ET{7i)+ of /i such as 

(xi,m^)= / {xi,y)dn{y), (12) 

for any Xi, X2,y & Ti.. A Gaussian measure ^ in ?i is a Borel measure such 
that for each xETi, there exist real numbers rUx and (Jx{> 0) satisfying 

ld{y G H; {y, x) < a] 

1 exp|=(^}*. 



■OO 



\/27rc7^ I 2(jJ 



The notation /i = [m, R] means that /i is a Gaussian measure on Ti with a 
mean vector m and a covariance operator R. 

Let {Hi,Bi) be an input space, (7^2, -82) be an output space and Pq"^ 
be the set of all Gaussian probability measures on (Hk,Bk){k = 1,2). We 
consider the case of (7ii,i3i) = (7^25-62) = (7i, S) for simplicity. Moreover, 
let fi e -P(7^) be a Gaussian measure of the input space and G -P(H) 
be a Gaussian measure indicating a noise of the channel. Then, a Gaussian 
channel A* from PiH) to PiH) is defined by the following mapping A : 
H X B ^ [0,1] such as 

/i(Q)=A>(g)= / A(x,Q)c/^(x) (14) 
Jh 

X{x,Q)^fiom, (15) 

Q" = {2/e7i:;Ax + yGg},xe7^,gGi5, (16) 

where ^4 is a linear transformation from TiioTi and A satisfies the following 
conditions: 

(1) \{x, ■) G PiTi) for each fixed x eH, 

(2) A(-, Q) is measurable function on (7i, B) for each fixed Q G -B. 



The compound measure $ derived form the input measure /i and the output 
measure ft is given by 

$(Qixg2)= / X{x,Q2)d^i{x) (17) 

for any Qi,Q2 G i3. 

In particular, let /i be [0,-R] G PiTi.) and /zq be [0,i?o] £ PO^)- Then, 
output measure A*/i = /i can be expressed as 

K*li = [0,ARA* + Rq]. (18) 

When the dimension of Ti is finite, the mutual entropy (information) with 
respect to /u and A* become ^ 

/(M;A-) = iiogl^^^§i^. (19) 

where | +Rq |, |/2o I are determinants of ARA* +i?o, -Ro- 
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5 ^-entropy and fractal dimension of a state 
for Gaussian measures in the random vari- 
able norm 

The £-entropy of states described in section 3 is different from Kolmogorov's 
definition of the £-entropy for random variable. In this section, we show that 
two definitions coincide when Ti. — R" and the norm of a state /U/ is defined 

by 



1 " r 

Then the distance between two states yU/ and jig induced by the above norm 
leads 



\ 



^E/J/.-..^,. (21) 

We call this norm random variable norm (R. V. norm for short) in the sequel. 
In this paper, we only consider Gaussian measures with the mean and 
Gaussian channels. 

Let an input state = [0, R] be induced from a n-dimensional random 
vector / = (/i, . . . , /„) and its output state A*/i/ be denoted by /ig, where 
g is random vector g = {gi, .... (?„) induced from A*. 

Lemmal: If the distance of two states is given by the above R.V. norm, 
then 

Jifif ; A*) = /(/x; ; A*) (22) 

proof: Prom the assumption, the Gaussian channel A* is represented by 
a conditional probability density of g with respect to / as 

p{y\x) 



(27r)t^^ 

X exp |-^(y - Ax)Ro\y - Axf^ 
x,ye R", 



where Ro is the covariance matrix associated to the channel A*. Then the 
compound state $ of /i/ and A*fif = fig is equal to the joint probability 
measure /^/^ = [0, C] of / and g such as 
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A*/ff(<5i X Q2) 

[ ^-^exp\-lzC-'Adz Qi,Q2eB(R''), 



where z is the 2n-dimensional random vector {x, y) = {xi,. . . ,Xn, yi, ■ . . , yn) 
and C is the following covariance matrix of fXfg ; 



C 



R RA* 
AR ARA^ + Ro 



(23) 



where R, RA\ AR, ARA^+Rq arenxn matrix and (i?)^ = E{fifj), {RA*)ij 
= E{figj), {AR)ij = Eigjj), {ARA'+Ro)ij = E{gigj) for each {ij 



l,...,n). 



For the channel T* satisfying A*///= r*/i/, T*jj,f is a n-dimensional Gaus- 
sian measure /x^ induced from a n-dimensional random vector h — {hi, 
. . . ,hn), SO that we have 

= sup r*); A>j = r>j} 

r* 

= sup {/(///; r);||AV;-rV;||=0} 
= sup {/(///; r*); - //ftll = 0} 



sup {/(/, h) 

h 



= sup 

h 



sup {/(/, /i) 

h 



E[d{g,hf] = Q} 

E[d(gi,hi)^]^0(i^l,...,n)} 
gi = hi a.e. {i = 1, . . . ,n)} . 



Prom gi — hi a.e. {i — 1, . . . ,n), we obtain 



Therefore, 



i{f,f,A*)^i{f,h)^i{f,f,n, 



(24) 
(25) 
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which imphes 

J(^;;A*) = /(/^;;A*). (26) 
Using the above lemma, the following theorem holds. 
Theoreml: Under the same assumption as Lemma 1 

(1) 5o(Ai/; e) = SkU; e) = \ Eti logmax l) , 

where Ai, . . . ,A„ are the eigenvalues of R and 6*^ is a constant uniquely 
determined by the equation I]"=imin(Aj,^^) = e^. 

(2) d^ifif) = n. 

proof: (1) Let C be the set of all Gaussian channels from B(RP') to 
i3(R"') and e) be the set of all Gaussian channels from i3(R") to BiTU") 
satisfying \\fif — A*/ij|| < e. According to Lemma 1, we obtain 

So{^lf■,e) = inf A*); A* gC(^j;£)} 

= inf{/(/i;;A*);A*GC(^/;e)} , 

From( we have 

= inf G 5(/i/;£)} 

= ^K(/;e), 

where S{fif, e) = [fifg] ^ Jj^n^-^n d{x,yyd^fg{x,y) < e}. 

The expression of the e-entropy | J2n=i log |niax (^^, ij | was obtained by 
Pinsker (1963). 

(2) Since 5'o(^/ ;e) = | E"=i log {max |, we have 

d^cM = rfJ^(^/) = limM^ 

° e 

^ Epilog {max (^,l)} 



lim - 

e^O log i 
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( 



i.e. ^ min(Ai, 9'^) — 



i=l 



n 



) 



= lim^ 



= lim - 



\ EILi log ^ 

\ Er=i log 

logi 



= n . 



In this case, our 5-entropy coincides with Kolmogorov's ^-entropy and the 
fractal dimension of the state /U/ is identical to the dimension of Hilbert 
space. 

6 ^-entropy and fractal dimension of a state 
for Gaussian measures in the total varia- 
tion norm 

In the following discussion, we only consider the case of dim(7i) = 1, that 
is, 7i = R, the state /x = [0, cr^] is a one-dimensional Gaussian measure 
(distribution) and the distance of two states is given by the total variation 
norm. 

In this section, we show that our e-entropy traces to the fractal property 
of a Gaussian measure but Kolmogorov's does not. The difference between 
5*0 and come from the norm of measures taken. We take the norm by 
the total variation, namely. 



Let Til = 7^2 = R- Then A becomes a real number f3 and the noise of a 
channel is exhibited by one-dimensional Gaussian measure /iq = [0, (Jg] G 
P(7i), so that the output state A*fi is represented by [0,P^a^ + ctq]. We 
calculate the maximum mutual entropy in the following two cases ; (1) /3^a"^-|- 
c^o ^ (2) P^cr^ + ctq < cr^ . Since the channel A* depends on /3 and (Tq, we 
put A* = ^*i3a^) ■ density function for Gaussian measures are error 

functions, we first give an order estimation for the difference of two Gaussian 
measures. 




(27) 
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Lemma2: If /9^a^ + ctq > a'^ and ||^ - A^^ ^2)/u|| = |^- A*^ ^2)/U,|(R) = 5, 
then 

(2){A(,,..);II/^-A(,,..)/^II<4 = 



Since (z = 1, 2) are even functions, we obtain 



where o(<5) is an order of S : hm^^o o{S)= and M{e) = {5 G R; < <5 < e}. 

proof: (1) Let ]?i,]?2 be the density functions of /i,A*/i, respectively. 
Then, we have 

(28) 



(29) 
(30) 
(31) 
(32) 



R 



\pi{x) -p2{x)\dx 



2 / \pi{x) - p2{x)\dx 
Jo 

2 (^y" (piix) - p2ix))dx + I {p2{x) - pi{x))dx 

4 / (pi(x) - p2(x))dx 
Jo 

I 



\ 



< 



< 



4a 



p^a^ + ai^ 
1 



+ 



a" 



277 \ a 



4 + 



(33) 



(34) 



where a is a real number satisfying pi{a) = ^2(0), and the first inequality 
is led by a geometrical approximation of the equation (^2]) and the second 
inequality is obtained from the inequality log a; < x — 1 for any positive 
number x. 

Since ||// — A*^^2)/u|| and are monotone decreasing for 
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P^a^ + ctq CT^ and 



lim Wfi- Air, 2m\\ 



4 V/?V2 + ag-cT 

lim 



0, 



the above inequality implies 



II/. - Af,,,2)/.|| = 5 ^ -= = 5 + 0(5), (35) 



where o{6) is an order of 6. 
(2) From Lemma2(l), 



4 y^/52(T2 + - a 
2^ a 



A(A.2); = 6 + o{S) } . (36) 



Let M{e) be the set of all 5 G R satisfying < 6 < e. From (^), it is clear 
that 



4 y/jV^ + qg - q 

27r a 



A(/3,.2);^^^ = 5 + 0(5), 6eM{e)} (37) 



Lemma3: Let A^^^^a^ be a channel satisfying /3 cr + > a and ||/i — 
A^Jj-^ ^2-)/u|| = S for any 5 G M(e). If a Gaussian channel AJ|-^^2) satisfies the 
condition (3'^ < ^^^i^, then we have 

^(/^; A^(,,.2)) = ilogi + i log (^1 + ^(5 + 0(5))) , (38) 
where Cg = (3^a^ + ctq is a constant determined by ||/U — A^^^ ^2)/u||= 5. 
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proof: The mutual entropy of /j, with respect to channel A* is 



/(/.;A*) = ilog^^^^. (39) 



Thus, if A^^^^2) is any channel satisfying — A^^^^2)A*|| = S, then we have 
from the above lemma 2 



The assumption implies, 
J(//;A*) 

= ^sup |/(//;A;^^^-gP;A*(^,^g)/. = A*(^^^-g^/.} 

= ^sup {/(/x;AJ^^^-2));||AJ(^_,2)/x-A*(^^^-2^ 

= sup |/(/x;AJ^^^-2p;/3V + a2 = ^V + ct2| 

where /J^ = ^ and = (l + + o(^)))^ 

Lemma^: Let A^|-^^2) be a channel satisfying /J^cr^ + ctq < o"^ and ||^ — 
A^i-^^2)/i|| = 5 for any 5 G M(e). If a Gaussian channel A^j^^^2) satisfies the 
condition < then we have 

A^^•. A(Aag)) = ^ log 1 + ^ log 7 (40) 
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where Cs = (3 a + (Tq is a constant determined by — ^sip a^)l^\\~ ^■ 

proof: Similarly proved as Lemma 2 and Lemma 3. 

Using the above those lemmas, we obtain the following theorem. 

Theorem2: Under the same conditions of Lemma 3 and 4, we have 

(1) Soifi; £) = I log i + i log ——^ ^ > 5k(//; e) = 

(2) d^if,) = I 

proof: (1) Prom Lemma 3 and 4, we have 
Soil^; e) = inf {J{i^; A*); - A*//|| < e} 

A* 

ilogi + iloga2(l + -^(5 + o(5)))' 



(l+^(5+o(5))) 



¥i |iog| + iiog- — — ; 5eM(£) 



1, 1 1, 



inf ^ -log- + -log- — -2 ; 5eM(£) 

5 12 J 2 (i + V^^s + o{5))) 



1, 1 1, 
2^°Ss + 2 



= -log- + -log 2' 

because ^ log t + k log ^ — is monotone decreasing with respect 

to 5 . 

(2) From (1), we obtain 



log^ 
ilogi + ilog^^^^ ^ 



lim , 

e-o log i 

1 

2 ■ 
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This result show that (1) the fractal dimension of a Gaussian measure ^ in 
the total variation norm describe fractal structure of Gaussian measures, and 
(2) The fractal dimension of a Gaussian measure /x is always if we use the 
Kolmogorov e-entropy. 

Wc concluded that our fractal dimension of states is a new criterion to 
study a chaotic aspect of Gaussian measures. 

References 

[1] AkASHI, S. (1992), A relation between Kolmogorov-Prokhorov's con- 
dition and Ohya's fractal dimensions, IEEE. Trans. Inf. Theory 38, 
1567-1570. 

[2] AkASHI, S. (1994), An entropy-theoretic classification of normal states. 
Open Systems and Information Dynamics 2, No. 3, 355-361. 

[3] BAKER, C.R. (1978), Capacity of the Gaussian channel without feed- 
back. Inform, and Control 37, 70-89. 

[4] GELFAND, I. M . AND YAGLOM, A. M. (1959), Calculation of the 
a mount of information about a random function contained in another 
such function, Amer. Math. Soc Transl. 12, 199-246. 

[5] INGARDEN, R.S., KOSSAKOWSKI, A. AND OHYA, M. (1997), "In- 
form ation Dynamics and Open Systems" , Kluwer. 

[6] KOLMOGOROV, A.N. (1963), Theory of transmission of information, 
Amer. Math. Soc. Translation, Ser.2, 33, 291-321. 

[7] KOLMOGOROV, A. N. AND TIHOMIROV, V. M. (1961), e-entropy 
and e-capacity of sets in function space, Amer. Math. Soc. Transmission, 
Ser. 2, 17, 277-364. 

[8] MANDELBROT, B.B. (1982), "The Fractal Geometry of Nature", W. 
H. Freemann and company, San Francisco. 

[9] MATSUOKA, T. AND OHYA, M. (1995), Fractal dimension of states 
and its application to Ising model. Rep. Math. Phy. 36, 365-379. 



16 



[10] OHYA, M. (1989), Some aspects of quantum information theory and 
their apphcations to irreversible processes, Rep. Math. Phys., 27, 19-47. 



[11] OHYA, M. (1991), Fractal dimension of states. Quantum probability and 
Related Topics VI, World Scientific, Singapore, 359-369. 

[12] OHYA, M. AND PETZ, D. (1993), "Quantum Entropy and its Use", 
Springer- Verlag. 

[13] PINSKER, M. S. (1963), Gaussian sources. Problems of Information- 
Transmission 14, 59-100 (in Russian). 

[14] YANAGI, K. (1988), An upper bound to the capacity of discrete time 
Gaussian channel with feedback. Lecture Notes in Math. 1299, 565-570. 



17 



